Database systems typically store database objects (e.g. tables, indexes, etc.) on disk, and load data items from those database objects into volatile memory on an as-needed basis. Once loaded into volatile memory, the data items may remain cached in volatile memory so that subsequent accesses to the same data items will not incur the overhead of accessing a disk. Those data items may be replaced in cache, for example, to make room in volatile memory to store other data items that have been requested.
Rather than load individual data items on a per-item basis, entire database objects, or portions thereof, may be pre-loaded into volatile memory. Various approaches for loading entire database objects, or selected portions thereof, into volatile memory to speed up query processing are described in U.S. patent application Ser. No. 14/377,179, entitled “Mirroring, In Memory, Data From Disk To Improve Query Performance”, filed Jul. 21, 2014, referred to herein as the “Mirroring” application, the contents of which is incorporated herein in its entirety.
According to the approaches described in the Mirroring application, database objects, or portions thereof, are stored in volatile memory in a different format than the format that those same objects have on disk. For example, the in-memory copies of the objects may be stored in a column-major format, while the on-disk copies are stored in a row-major format. An in-memory version or copy of an object (or selected portions thereof), is referred to herein as an In-Memory-Copy (IMC). The set of data that is copied from disk into volatile memory to create an IMC is referred to herein as a “chunk”.
In a database cluster, when one node in the cluster loads a chunk from disk into volatile memory, other nodes are prevented from making changes to the chunk to keep the data consistent within the IMC. This can be accomplished by having the node that is loading the chunk obtain an exclusive lock to lock the chunk before loading the chunk. The exclusive lock on the chunk can be released when the chunk has been fully loaded into volatile memory. While the chunk is momentarily locked during the pre-loading operation, data items from the chunk cannot be changed (such as created, updated, or deleted) through data manipulation language (DML) operations.
Unfortunately, for applications that require high volumes of DMLs, such as online transaction processing applications, locking entire chunks while the chunks are pre-loaded would cause the application to experience significant interruptions. Blocking DMLs from executing during chunk loading is an unacceptably high cost. On the other hand, granting DMLs a higher priority than chunk-loading may prevent the load operation from ever finishing.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.